Skip to content

Add IODA comparisons#798

Open
mranst wants to merge 18 commits into
developfrom
feature/mranst/hofx_comparisons
Open

Add IODA comparisons#798
mranst wants to merge 18 commits into
developfrom
feature/mranst/hofx_comparisons

Conversation

@mranst

@mranst mranst commented Jun 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Description

Add comparisons to hofx observations. Used to evaluate differences for observations. Allows comparing hofx suites, and comparison plots for all suites will only be run if a difference is detected for each individual obs. This works by reading the output obs file and averaging values along a set length to check if they are equivalent between the two experiments. I implemented it this way for memory concerns, we could switch to evaluating for point-by-point differences if that makes more sense.

@mranst mranst marked this pull request as ready for review June 15, 2026 20:18
@mranst

mranst commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator Author

I had to fight cylc to get this to work as intended, I was trying to get it to operate on each observation individually, but it would choose to wait for requisite tasks that were never triggered. I switched it back operating as one task.

@mranst mranst requested review from Dooruk, jeromebarre and mer-a-o June 15, 2026 21:23
@Dooruk

Dooruk commented Jun 16, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Sorry, quick question. Is the main idea of this PR would be using hofx0 values if it's a variational and/or FGAT suite and use hofx if it's an hofx type suite (hofx or hofx_cf)?

@mranst

mranst commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator Author

Sorry, quick question. Is the main idea of this PR would be using hofx0 values if it's a variational and/or FGAT suite and use hofx if it's an hofx type suite (hofx or hofx_cf)?

Correct, is that sensible?

@Dooruk

Dooruk commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Correct, is that sensible?

Yeah. Maybe not necessarily for this PR but I was just wondering instead of hofx or hofx0 only if they could be chosen from a list from the following IODA outputs :

group: EffectiveError0 {
  variables:
        float absoluteDynamicTopography(Location) ;
                absoluteDynamicTopography:_FillValue = -3.368795e+38f ;
  } // group EffectiveError0

group: EffectiveError1 {
  variables:
        float absoluteDynamicTopography(Location) ;
                absoluteDynamicTopography:_FillValue = -3.368795e+38f ;
  } // group EffectiveError1

group: EffectiveQC0 {
  variables:
        int absoluteDynamicTopography(Location) ;
                absoluteDynamicTopography:_FillValue = -2147483643 ;
  } // group EffectiveQC0

group: EffectiveQC1 {
  variables:
        int absoluteDynamicTopography(Location) ;
                absoluteDynamicTopography:_FillValue = -2147483643 ;
  } // group EffectiveQC1

group: MetaData {
  variables:
        int64 dateTime(Location) ;
                dateTime:_FillValue = -9223372036854775801LL ;
                string dateTime:units = "seconds since 1970-01-01T00:00:00Z" ;
        float latitude(Location) ;
                latitude:_FillValue = -3.368795e+38f ;
                string latitude:units = "degrees_north" ;
        float longitude(Location) ;
                longitude:_FillValue = -3.368795e+38f ;
                string longitude:units = "degrees_east" ;
  } // group MetaData

group: ObsBias0 {
  variables:
        float absoluteDynamicTopography(Location) ;
                absoluteDynamicTopography:_FillValue = -3.368795e+38f ;
  } // group ObsBias0

group: ObsBias1 {
  variables:
        float absoluteDynamicTopography(Location) ;
                absoluteDynamicTopography:_FillValue = -3.368795e+38f ;
  } // group ObsBias1

group: ObsError {
  variables:
        float absoluteDynamicTopography(Location) ;
                absoluteDynamicTopography:_FillValue = -3.368795e+38f ;
                string absoluteDynamicTopography:units = "m" ;
  } // group ObsError

group: ObsValue {
  variables:
        float absoluteDynamicTopography(Location) ;
                absoluteDynamicTopography:_FillValue = -3.368795e+38f ;
                string absoluteDynamicTopography:units = "m" ;
  } // group ObsValue

group: PreQC {
  variables:
        int absoluteDynamicTopography(Location) ;
                absoluteDynamicTopography:_FillValue = -2147483643 ;
  } // group PreQC

group: hofx0 {
  variables:
        float absoluteDynamicTopography(Location) ;
                absoluteDynamicTopography:_FillValue = -3.368795e+38f ;
  } // group hofx0

group: hofx1 {
  variables:
        float absoluteDynamicTopography(Location) ;
                absoluteDynamicTopography:_FillValue = -3.368795e+38f ;
  } // group hofx1

group: oman {
  variables:
        float absoluteDynamicTopography(Location) ;
                absoluteDynamicTopography:_FillValue = -3.368795e+38f ;
  } // group oman

group: ombg {
  variables:
        float absoluteDynamicTopography(Location) ;
                absoluteDynamicTopography:_FillValue = -3.368795e+38f ;
  } // group ombg
}

@mranst

mranst commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator Author

Yeah. Maybe not necessarily for this PR but I was just wondering instead of hofx or hofx0 only if they could be chosen from a list from the following IODA outputs :

Done. Now goes through a list for those variables. I have it set to hofx for hofx suites, and hofx0 and hofx1 for variational, but it should be able to support all of them

@mranst mranst changed the title Add hofx comparisons Add IODA comparisons Jun 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Dooruk Dooruk Awaiting requested review from Dooruk

@mer-a-o mer-a-o Awaiting requested review from mer-a-o

@jeromebarre jeromebarre Awaiting requested review from jeromebarre

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@mranst @Dooruk